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PREFACE 


During  a summer  workshop  (funded  by  the  Sloan  Foimdation) 
that  we  ran  at  Yale,  I tried  to  present  some  of  the  views  that  we 
hold  with  respect  to  the  problems  of  representation  of  meaning, 
the  making  of  Inferences,  and  the  function  of  higher  level 
descriptions  of  the  structure  of  knowledge,  to  an  audience 
primarily  consisting  of  social  and  cognitive  psychologists.  Most 
of  the  participants  In  the  workshop  were  Interested  In  our  Ideas 
on  this  subject.  However  their  background  really  had  not 
prepared  them  to  understand  why  we  did  what  we  did  or  how  we  came 
to  do  it.  Consequently,  I attempted  to  give  them  that  background 
by  retracing  the  steps  in  our  research  of  the  last  ten  years.  I 
explained  how  we  came  to  hold  our  current  views  on  various 
snb1e<  t8  by  showing  wliat  our  Initial  assumptions  were  and  how  one 
pontlon  naturally  led  us  to  the  next.  Since  most  of  the 
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participants  In  the  workshop  felt  that  these  lectures  provided 
the  context  that  was  necessary  to  help  them  to  understand  our 
current  research,  I felt  that  It  would  be  of  use  to  prepare  a 
paper  based  on  the  lecture  notes  that  I used  In  the  workshop# 


This  paper  therefore  Is  necessarily  sketchy*  It  Is  an 
attempt  to  present  the  outline  of  ten  years  of  research  and  as 
such  can  only  barely  cover  the  Issues.  Furthermore,  the  paper  Is 
entirely  biased  towards  the  research  within  our  own  group.  Much 
other  work,  some  well  known  and  other  less  known  occurred  before 
and  during  the  work  described  here.  To  some  extent  this  work 
affected  our  own,  but  by  and  large  the  work  described  here 
proceeded  on  its  merry  way  untouched  by  very  much  from  the 
outside*  In  this  paper  I shall  attempt  to  show  how  and  why  we 
got  to  where  we  are  today. 


1966-1969 


My  initial  research  focussed  on  the  representation  of 
meaning  as  It  would  be  used  for  the  generation  of  natural 
language  sentences.  Since  generation  was  the  major  problem  In 
both  linguistics  and  computational  linguistics,  this  point  of 
departure  was  not  particular  different  from  the  established 
norms.  The  major  difference  was  that  my  representations  were 
Intended  to  bo  psychologically  correct  (to  the  extent  that  that 
could  be  determined).  This  led  me  avjay  from  what  I believed  to 
be  the  many  ad  hoc  entitles  that  existed  within  transformational 
deep  structures  at  the  tlme.*l* 


J 
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I thus  began  to  think  about  the  problem  of  representing 


*1*  I started  out  In  Artificial  Intelligence  as  an  undergraduate 
In  mathematics  (vhich  Included  computer  science)  at  Carnegie  Tech 
(now  Carnegle-Mellon)  In  the  early  sixties*  I became  Interested 
In  language,  and,  feeling  that  language  would  best  be  studied 
with  those  whose  specialty  It  was,  I went  to  study  Linguistics* 
When  I arrived  at  the  University  of  Texas  in  1966,  Texas  was  a 
combination  of  two  relatively  unrelated  paradigms* 
Transformational  Generative  grammar  was  beginning  to  become  the 
dominant  paradigm  within  the  department  of  Linguistics  Itself* 
But,  at  the  same  time,  an  extremely  large  mechanical  translation 
project  that  had  flourished  at  Texas  for  some  time,  was  present 
but  drawing  to  a close,  at  least  partially  due  to  the  ALPAC 
report  which  was  Issued  that  year*  Tills  meant  that  while  there 
were  a largo  number  of  computer  oriented  people  In  Austin  whose 
Interest  was  language,  the  power  In  the  department  itself,  where 
I was  a student,  was  held  by  the  transformationalists* 

As  a computer  type  myself,  I naturally  assumed  that  the  main 
issue  in  language  study  ought  to  be  the  representation  of 
meaning*  Not  only  was  this  not  the  view  held  by  either  of  the 
paradigms  present  In  Austin  at  the  time,  but  it  was  a view  whose 
opposite  position  was  strongly  held  by  all  those  around  me* 
Syntax  was  the  Issue  of  the  day  in  the  Linguistics  department 
itself,  and  at  the  Linguistics  Research  Center  (LRC)  where  the  MT 
work  was  being  done,  gigantic  phrase  structure  grammars  and  rules 
that  transfered  structure  from  one  language  to  another  were  what 
was  being  done* 

My  views  came  to  me  by  way  of  Newell  and  Simon  at  Carnegie; 
(Actually  this  was  quite  Indirect*  I had  never  met  either  of 
them,  but  their  work  on  GPS  and  more  Importantly  their  views  on 
the  nature  of  computer  programs  as  theories  was  well  known  to 
most  students  at  Carnegie)  , by  Sheldon  Klein  who  was  my  teacher 
at  Carnegie  and  by  Sydney  Lamb  who  was  Klein's  teacher  and  whom  I 
had  met  and  Interacted  with  at  the  Linguistics  Institute  held  at 
the  University  of  Michigan  In  the  summer  preceding  my  senior  year 
at  Carnegie* 

My  own  view  represented  a sort  of  amalgamation  of  the  vlevrs 
of  all  these  people*  I was  encouraged  In  my  effort  by  my  advisor 
at  Texas,  Jacob  Mey,  and  by  my  employer  at  Tracer  ( a company 
that  allowed  some  of  the  LRC  people  to  move  there  after  the  LRC 
funding  dried  up)  Eugene  Pendergrnft* 
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ipeanlng.  In  particular,  as  I was  still  interested  in  the 
computability  of  any  representation  that  I came  up  with,  I was 
especially  concerned  with  the  question  of  how  a meaning 
representation  could  be  of  use  in  the  generation  of  natural 
language  sentences,  and  in  the  parsing  of  natural  language 
sentences. 

The  first  representation  that  I came  up  with  looked  a lot 
like  English  with  arrows  connecting  it  up.  The  arrows  were 
gotten  from  dependency  theory  which  had  been  written  about  by 
Hays  and  used  quite  a bit  by  Klein  and  to  some  extent  Lamb.  My 
contribution,  as  I saw  it  at  that  time,  was  to  make  the 
representation  more  conceptual.  I reasoned  that  the  dependency 
grammars  being  used  at  that  time  were  too  concerned  with 
questions  of  whether  the  noun  or  the  verb  was  really  the  head  of 
the  sentence  and  not  enough  concerned  with  the  meaning  of  the 
sentence.  Obviously,  the  main  noun  and  main  verb  contributed 
equally  to  the  conceptualization  that  underlied  the  sentence. *2* 
That  is,  both  were  necessary  for  the  sentence  to  have  meaning.  I 
concentrated  on  Issues  such  as  these,  coming  up  in  the  end  with 
rules  that  described  the  make  up  of  a conceptualization  in  terms 
of  items  more  conceptual  than  nouns  and  verbs. 

I then  began  to  work  on  the  problem  of  how  my  representation 
system  would  allow  for  random  generation  of  English  sentences 
(this  being  my  computer-biased  view  of  the  work  in  Linguistics  at 
the  time  and  thus  the  field  in  wfiich  I saw  myself  doing  battle) 

*2*  That,  of  course,  is  the  origin  of  the  symbol  <»>. 
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and  hov  my  system  would  work  for  the  representation  of  languages 
other  than  English. *3* 

To  facilitate  the  former  of  these  tasks,  I had  to  invent 
what  I called  "a  conceptual  semantics"  (Schank,  1968)  which  was 
basically  a depository  of  world  knowledge  which  prevented  any 
random  generator  from  generating  sentences  which  meant  nothing. 
The  latter  considerations  of  universality  caused  various  English 
items  such  as  prepositions  to  drop  out  of  the  representations. 
Even  so,  my  representations  bore  a great  deal  of  similarity  to 
the  surface  properties  of  English.  I was  aware  of  this  problem, 
but  was  far  more  concerned  at  the  time  with  attempting  to 
convince  linguists  that  meaning  considerations  were  Important .*4* 

At  this  point  I was  rather  anxious  to  get  a job,  and  largely 
through  chance,  found  myself  employed  by  Kenneth  Colby  at 
Stanford. *5*  My  job  within  his  project  was  to  create  a parser 
that  would  allow  his  soon  to  be  created*6*  version  of  an 
automated  psychiatrist  to  actually  understand  what  the  patient 
said.  To  do  this,  I attempted  to  reverse  the  rules  that  I had 
already  written  for  generation  out  of  Conceptual  Dependency  (CD). 


*3*  My  system  being  conceptual  It  ought  to  have  been  capable  of 
handling  this  latter  task.  Furthermore,  I could  hardly  have  not 
worried  about  this  Issue  since  the  emphasis  in  Linguistics  was  on 
what  were  then  called  "funny  languages"  (In  my  case  I worked  on 
Quiche  and  a little  on  Eskimo)  and  because  MT  was  clearly  In  need 
of  an  Interllngua  that  would  facilitate  translation. 

*4*  I now  know  that  there  were  some  who  were  already  convinced  of 
that  fart  but  they  were  not  at  Texas  and  were  unknown  to  me. 
Fillmore's  work  was  discussed  but  its  properties  were  still  far 
more  syntactic  than  I had  In  mind. 
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Such  a reversal  of  rules  was  established  doctrine  within 
computational  linguistics  and  I assumed  It  would  work.*?* 

As  an  example  of  the  kind  of  Issue  I was  concerned  about  at 
that  time  consider  the  sentences; 


I hit  Fred  on  the  nose 
I hit  Fred  In  the  park 

In  order  to  parse  these  sentences  correctly  it  Is  necessary  to 
know  where  a person  can  be  located.  Here,  "correctly"  depended 
on  what  had  to  be  represented  in  CP.  There  was  a locative  for 
entire  conceptualizations  and  a "part  of"  relationship  for 
objects,  and  either  could  be  expressed  In  English  with  a locative 
prepositional  phrase.  To  solve  this  problem  I used  the 
conceptual  semantics  I had  invented  for  generation,  and  my  rules 
that  mapped  from  syntactic  relationships  to  conceptual  ones 
checked  for  acceptability  each  time  a mapping  was  attempted.  The 


*5*  Colby  had  at  this  point  recognized  that  his  dreams  of  an 
automated  psychiatrist  depended  on  solving  the  natural  language 
problem  first.  I of  course  agreed  with  this  and  found  the  idea 
of  an  automated  psychiatrist  fascinating.  I had,  until  this 
time,  never  really  thought  about  the  higher  level  processes  that 
V’ere  to  operate  on  top  of  any  meaning  representation  I came  up 
with.  I was  Interested,  prior  to  my  working  for  Colby,  mostly  in 
mechanical  translation.  Shortly  after  I arrived  Colby  and  his 
project  moved  physically  to  the  Stanford  AI  lab,  thus  cementing 
my  Involvement  with  AI. 

*6*  It  never  actually  existed  in  Its  planned  newer  form. 

*7*  Of  course,  there  were  no  semantic  representations  in 
computational  linguistics  at  that  time,  those  Ideas  about 
reversal  had  to  do  with  syntactic  rules.  I saw  no  reason  to  not 
go  directly  to  my  new  conceptual  base,  and  an5rway  as  I have  said 
my  conceptual  base  looked  an  awful  lot  like  English  in  any  case, 
so  this  did  not  scorn  to  be  that  great  a shift. 


same  thing  was  necessary  for  sentences  siicVi  as: 


I hit  the  boy  with  long  hair 

However , what  had  to  be  done  to  handle  ambiguous  sentence,  was  to 
add  information  In  the  conceptual  semantics*  The  conceptual 
semantics  for  'hit'  consisted  of  Information  about  the  kind  of 
objects  to  be  found  in  various  prepositional  relationships.  So, 
for  'hit'  we  had: 

with  - weapon  object 
by  - no 

on  - part  of  < — PP 

The  final  parse  of  this  sentence  put  this  additional  Information 
concerning  the  properties  of  "hair"  In  the  actual  representation. 
Although  I did  not  use  the  term,  this  was  in  a sense  the  first 
class  of  Inference  to  be  made,  and  added  to  the  meaning 

representation  In  my  work.  Since  it  had  to  be  determined  If  the 

'with'  object  for  'hit'  was  a weapon  or  a part  of  the  object, 

that  determination,  once  made,  now  became  part  of  what  had  been 
understood  and  thus  was  part  of  what  was  meant.  The  can  of  worms 
that  this  adding  of  Information  not  actually  stated  opened  was 
tremendous.  Additional  Information  could  be  added  to  what  had 
been  said  to  form  what  had  been  meant.  This  was  quite  different 
from  what  had  gone  on  in  linguistics  up  until  that  time  (and  to 
some  extent  what  still  goes  on  In  linguistics).  I was  saying 
that  the  meaning  of  a sentence  was  more  than  the  sum  of  Its 

parts.  This  heresy  was  not  particularly  appreciated  when  I 
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brought  it  up,  although  it  probably  had  seemed  obvious  to  those 
AI  people  that  had  looked  at  the  problem. 

Thus,  my  point  was  that  Qiomsky  was  wrong  In  claiming  that 
we  should  not  be  attempting  to  build  a point  by  point  model  of  a 
speaker-hearer.  Such  a model  was  precisely  what  I felt  should  be 
tackled.  Linguists  viewed  this  as  performance  and  thus 
uninteresting.  I took  my  case  to  psychologists  and  found  them 
equally  uninterested.  Psychologists  Interested  in  language  were 
mostly  psycholinguists,  and  psycholinguists  for  the  most  part 
bought  the  assumptions  of  transformational  grammar  (although  it 
seemed  very  odd  to  me  that  given  the  competence/performance 
distinction,  psychologists  should  be  on  the  side  of  competence). 

1970 


In  1970  we  started  to  make  our  representations  more 
conceptual .*8*  Until  this  point  our  supposedly  language-free 
representations  had  a great  deal  of  language  in  them.  We  noticed 
a class  of  verbs  (vrtilch  we  termed  pseudo-state  verbs),  where  the 
object  of  the  sentence  did  not  seem  to  be  the  same  as  the 
underlying  deep  object  of  the  underlying  action.  In  particular, 
our  representations  seemed  to  require  us  to  put  in  a great  deal 
more  than  was  in  the  sentence  in  order  to  make  conceptual  sense. 
Thus  for  (1)  below  we  had  to  make  up  something  called  'create'. 


*8*  The  'we'  consisted  of  Larry  Tesler  (a  programmer  who  worked 
for  Colby  and  was  attempting  to  write  our  parser)  and  Sylvia 
Weber  (a  graduate  student  in  the  Computer  Science  department  at 
Stanford .) 
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In  (2)  we  had  a particular  sense  of  'have'  and  In  (3)  all  of  a 
sudden  we  had  'truth'  and  'saying'  present.  There  did  not  seem 
to  be  any  way  to  avoid  this  Introduction  of  elements  that  weren't 
there  Initially  If  we  were  to  represent  the  meaning  of  what  had 
been  said: 

1)  he  wrote  a book 

by  he 

he<“=>create< — book< — 

II 

writes 

2)  he  desired  Martha  In  the  morning 

he<=**>want 

he<=«=>have< — Martha 

I 

morning 

3)  he  doubted  his  wife 

he<">doubt 

x<"““>true 

I 

\ / 

wife  <“>  say 
of 

II 

he 

Examining  these  representations,  we  began  the  search  for  some 
regularities  in  the  representation  that  would  give  us  a more 
canonical  form.  What  we  had  until  that  point  was  so  free  form 
that  wo  could  create  anything  at  any  time.  This  did  not  seem 
very  senslhle.  Tn  particular,  there  was  a problem  of  idiat  sense 
of  the  various  multiple  sense  verbs  we  had  at  any  given  time.  We 
couldn't  .lust  continue  writing  'have'  the  way  we  had  done.  There 
had  to  have  been  some  underlying  basic  forms.  We  considered  for 


a while  Just  writing  subscripts  on  the  verbs.  co  'understand!' 
was  equal  to  'see^'.  But  which  sense  was  more  basic?  And,  more 
Importantly,  how  many  senses  of  a word  would  there  turn  out  to  be 
and  vdiat  would  their  Intersections  be?  In  the  case  of  partial 
overlap  of  senses  there  was  a definite  problem  with  the  subscript 
method . 

As  a side  Issue  at  this  time,  we  attempted  to  clean  up  the 
mess  In  which  we  had  left  our  representation  of  prepositions.  We 
had  been  using  an  arrow  to  mean  any  prepositional  relationship, 
in  the  faith  that  higher  level  processes  that  used  our 
representations  (we  really  had  no  idea  what  they  would  be  like 
the  psychiatric  model  would  have  to  deal  with  that  problem)  would 
figure  out  the  true  relationship  that  held  between  an  action  and 
Its  associated  objects.  We  tried  to  think  about  what  kinds  of 
prepositional  relationships  there  were.  Location  had  long  before 
been  relegated  to  describing  conceptualizations  themselves,  so  It 
wasn't  a candidate.  'Part  of'  relationships  were  used  to 
describe  objects,  rather  than  the  relationship  between  actions 
and  objects  so  they  weren't  part  of  the  problem  either.  By 
eliminating  these  two  classes  of  prepositions  (those  that 
described  entire  events  and  those  that  solely  described  objects) 
we  found  that  there  were  only  three  kinds  of  preposltinal 
relationships:  Instrumental,  directional,  and  recipient.  These 
relationships  then  described  the  way  an  action  could  relate  to  an 
object  in  an  event  regardless  of  vhat  preposition  was  being  used. 
Since  we  were  describing  relationships  and  not  prepositions,  we 
realized  that  English  could  be  considered  to  have  a kind  of  null 
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preposition  denoting  objective  relationships  but  that  this  did 
not  Indicate  that  this  was  any  less  of  a relationship  between 
action  and  object  than  the  others.  We  knew  that  Fillmore  had 
said  similar  things  about  syntactic  relationships  In  English  so 
we  christened  our  relationships  'conceptual  cases'.  The 
differences  between  the  two  systems  were  a lot  greater  than  their 
names  suggested  and  in  retrospect  this  was  probably  a poor  choice 
of  names  (see  Schank  (1972)  for  a discussion  of  those 
differences) . 

This  new  system  of  cases  Immediately  had  ramifications 
throughout  our  entire  system.  Thus,  for  example  we  had 
previously  represented  'I  want  money'  as: 

I <«>want 

to 

I _ 

money  <“«>go  < 1 

However,  adding  a recipient  to  this  representation  caused  us  to 
come  up  with  the  following  representation: 

I <«>=>want 

I 0 R I >I 

Someone  <*>  ??  < — iPoney< — | 

I <someone 

That  is,  we  knew  that  we  had  a Recipient  here  and  it  had  to  be 
'I'.  Similarly  there  had  to  be  an  Object  because  what  else  could 
'money'  be?  It  didn't  seem  like  an  actor.  The  actor  was  unknown 
but  we  knew  he  was  the  same  person  as  the  donor  of  the  recipient 
cose.  Of  course,  the  above  diagram  had  a glaring  hole.  IThat  was 
the  action?  Still  this  rrpresentalon  seemed  to  make  a lot  more 
sense  than  the  one  having  money  be  an  actor  that  did  the  action 


i 
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'fo' .*9*  What  was  neorted  at  this  point  was  a nano  for  our  unknown 
action,  and  since  It  was  obviously  a kind  of  transfer  of  the 
money  that  was  being  done  we  called  It  'trans'. 

'Trans'  helped  us  with  other  problems  as  well.  It  solved 
the  partial  overlap  problem  In  the  meaning  of  words  such  as 
'give'  and  'take'  and  'buy'  and  'sell'.  Furthermore,  It 
eliminated  the  need  for  elaborate  transfer  of  meaning  rules  of 
the  kind  Katz  (1967)  had  been  proposing  for  mapping  words  like 
'buy'  Into  'sell'.  We  began  to  wonder  what  other  actions  like 
'trans'  were  around. 

We  began  to  look  at  other  representations  we  had,  using  the 
conceptual  case  notions  that  we  had  Invented.  For  example  we  had 
represented  'He  heard  me'  as: 

I 

he<*>percelve< ears 

of 

II 

be 

& >he 

I <->8ay  < X< I 

<I 

'Hear'  obviously  took  a conceptual  object  of  sound  that  got 
translated  Into  a meaning  by  some  process.  'Me'  was  not  a 
sensible  conceptual  object  for  'hear'.  Similarly  the  conceptual 
form  of  'hear'  needed  an  Instrument  that  had  something  to  do  with 


*9*  I should  point  out  that  one  of  the  basic  maxims  of  CD  was 
that  there  was  an  actor-action-object  framework  Into  which  things 
should  fit.  This  was  part  of  the  rules  that  I worked  out  in  my 
thesis  (Schank,  1969). 
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'earn' • S»trli  r onstd era t Iona  I nri-c'd  ns  to  roarrnnRf  i lio  Hcntoocp 
Into  a conceptual  format  that  represented  the  fact  that  the 
meaning  of  an  Idea  being  transferred  was  the  key  action  taking 
place* 


We  began  at  this  point,  to  look  more  closely  at  the  concept 
of  an  action*  We  attempted  to  classify  the  verbs  we  had  been 
using  according  to  the  cases  they  took  and  properties  of  thler 
objects*  This  left  us  with  S(tate)-'ACT8 , Plhyslca!  j-ACTs , 
E (root Iona 1) -ACTS , and  so  on  (SchanV  et  al*,1970)*  Using  this 
classification  for  verbs,  we  could  now  predict  the  missing  cases 
that  were  implicit  and  thus  tliat  had  to  be  Inferred*  We 
continued  to  look  for  effective  gouplngs  that  would  facilitate 
Inference*  Thus,  while  we  did  not  actually  set  out  to  discover 
primitives,  the  considerations  that  we  had  in  representation 
Issues  forced  us  to  come  up  with  some  workable  classification  of 
actions  that  fit  within  our  framework* 

Inference  was  not  yet  a major  Issue  In  this  regard,  but 
other  problems  forced  us  to  focus  on  It*  For  example  consider 
the  sentence  'I  fear  boars'  and  our  proposed  representation  of  it 
at  that  time: 


I<“>fear 

A 

I 

beaT8<*>barm<~I 

In  the  same  paper  idiere  we  were  wrestling  with  the  Issue  of 
representation  of  actions  (Schank  et  al*,1970)  we  also  Introduced 
an  Idea  we  called  "associative  storage  of  concepts"* 


In  order  to 


adequately  represent  sentences  of  the  above  type,  it  was 
necessary  to  have  available  a conceptualization  that  could  serve 
as  the  object  of  the  verb  'fear'.  (At  this  point  we  viewed  such 
a verb  as  a kind  of  statlve  ACT.  We  later  realized  states  were 
not  ACTS  but  states  of  objects.)  Obviously  this  conceptualization 
had  to  have  in  it  both  'bears'  and  'I'  as  part  of  the  object  of 
'fear'.  Here  again  we  were  faced  with  the  question  of  what  was 
the  ACT?  The  answer  we  chose  was  an  ACT  called  'harm'.  As  we 
were  not  interested  In  primitives  particularly  this  should  not 
seem  strange.  The  focus  of  our  Interest  was:  how  were  we  going 
to  find  the  concept  'harm'  to  add  to  our  representation? 

The  answer  was  through  associative  storage  of  concepts. 
What  we  meant  by  this  was  that  there  had  to  be  some  connection 
between  fear  and  bears  that  would  allow  us  to  infer  'harm'  as  the 
missing  ACT.  Qulllian  (1966)  had  used  an  idea  of  a linked 
network  of  concepts  that  could  be  searched  from  two  paths  in 
order  to  find  their  shortest  intersection.  This  idea  had  been 
used  for  disambiguation,  but  it  now  seemed  that  it  could  be 
extended  for  use  here  as  well. 

However,  that  seemed  like  a lot  of  work  for  so  little.  When 
we  looked  at  other  examples  of  the  phenomenon  we  were  trying  to 
account  for,  an  easier  solution  presented  Itself.  For  example, 
the  sentence  'T  like  books'  clearly  needed  something  about  'I 
read  bonks'  Inside  the  conceptualization  that  represented  Its 
meaning.  Tt  was  obvious  that  this  could  he  done  simply  by 
ttsling  'books'  In  the  dictionary  ns  a 'READ  object'.  All  that 
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would  then  be  required  was  an  enpty  slot  requiring  an  ACT.  CD 

gave  us  that  when  a statlve  ACT  was  recognized,  so  all  we  had  to 

do  at  that  point  was  to  look  In  the  dictionary  for  an  ACT 

associated  with  the  object  we  had  available*  Ihls  did  not  solve  j 

the  problem  when  the  object  was  not  the  source  of  the  Inference  A | 

s 

functional  object  like  a 'book'  could  well  be  listed  as  a 'READ 
object',  but  what  wore  we  to  do  when  'bears'  or  'Nixon'  was  the 
object  of  a statlve  ACT?  Since  these  objects  were  not  functional 

i 

In  the  same  way.  It  seemed  that  the  missing  ACT  would  have  to  be 
supplied  as  a part  of  the  meaning  of  the  word  'fear'.  Here 
again,  we  had,  without  quite  Intending  to,  decomposed  the  meaning 
of  a word  (fear)  Into  more  basic  elements  (fear  plus  expected 

harm).  The  reason  this  had  happened  was  again  attributable  to  j 

the  requirements  we  had  put  on  CD  with  respect  to  %ihat  slots 

! 

there  were  in  a conceptualization  and  how  they  were  to  be  filled.  ^ 

I 

so,  we  were  left  at  this  point  td.th  a representation  like: 

I fear  Nixon 
I<«>fear 

I 

Nlxon<“>do 


something  <«*>harm< 1 


Thus,  at  this  point  ve  were  now  freely  adding  to  our 
representation  concepts  that  %ierc  not  present  In  the  English 
sentence  In  the  first  place,  and  perhaps  more  Importantly, 
concepts  that  wre  only  probably  part  of  the  meaning.  These  were 
the  first  explicit  Inferences  that  wp  had. 


I 
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1971 

In  1971,  we  began  to  focus  on  the  problem  of  the  Inference 
of  Intentions.  We  got  Into  this  problem  because  of  a peculiar 
use  of  language  that  we  happened  to  come  across,  that  we  realized 
It  was  crucial  for  an  reasonable  understanding  system  to  handle. 
The  example  was: 

Q:  Do  you  want  a piece  of  chocolate? 

A:  I just  had  an  Ice  cream  cone. 

Clearly,  It  is  necessary  to  understand  the  answer  here  as  meaning 
'no'.  In  attempting  to  figure  out  how  to  do  this,  we  realized 
that  It  was  necessary  to  fill  out  the  structure  of  the 
conceptualizations  underlying  both  sentences  so  that  a match 
could  be  made  from  the  answer  to  the  question.  To  do  this 
required  Inferences,  ones  that  were  different  from  the  "fill  In 
the  ACT"  ones  we  had  been  working  on.  Thus  we  needed  a structure 
like: 

want 

III 

trans 

III 

eat 

III 

satisfied 


To  gel  this  structure  we  had  to  postulate  that  when  a trans  was 
present,  the  object  being  transed  might  enable  an  actor  to 
perform  the  usual  functional  ACT  done  to  this  object. 
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Furthermore  we  had  to  examine  the  result  of  this  action,  because 
whatever  state  It  caused  was  the  key  for  the  pattern  match.  That 
Is,  a paraphrase  of  this  question  might  be  'Do  you  want  me  to 
trans  you  an  object  which  Is  edible  so  you  can  eat  It  so  that 
will  make  you  feel  some  feeling  (full,  happy  etc.)?  The  answer 
would  then  be  'I  already  have  that  feeling  because  1 just  did  an 
action  (here  'had'  has  to  be  Inferred  to  be  'eat')  that  resulted 
In  that  feeling.  To  do  all  this  required  a new  set  of 
resultatlve  and  enabling  Inferences,  and  began  to  cause  us  to 
focus  on  the  question  of  what  kinds  of  Inferences  there  were  and 
where  they  came  from. 

One  of  the  first  Issues  however  was  the  potential  use  of 
such  Inferences  however.  Since  we  were  primarily  concerned  with 
parsing  at  this  stage,  we  focussed  initially  on  the  Issue  of  what 
expectations  there  were  in  processing  that  came  from  places  other 
than  the  CD  or  syntactic  expectations  themselves. *10* 

We  looked  at  an  example  of  a conversation  where  a person  In 
a fit  of  anger  at  his  wife,  asks  for  a knife  from  a friend  and 
wlien  he  is  refused  It  says: 

I think  I ought  to... 

The  question  we  asked  was:  what  different  kinds  of  things  do  you 


*10*  Expectations  were  the  key  idea  behind  how  our  parser  was 
supposed  to  work.  The  parser  Tesler  designed  (Schank  and  Tesler, 
1969)  was  Intended  to  'laugh'  at  the  sentence  'I  saw  the  Grand 
Canyon  flying  to  New  York'  because  It  would  have  had  Its 
expectations  violated.  Rlesbeck  (1975)  of  course  later  did  this 
In  a more  serious  iwy. 
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expect  at  this  point?  V/e  Isolated  these  (Schank,  1971); 

1.  sentential  - a verb  coming 

2.  conceptual  - a entire  conceptualization  Is  coming 

3.  context  - "ought  to  have  fish" 

excluded  by  fighting  context 
something  violent  expected 

4.  conversational  - Inference  of  reason  person  Is  talking 

why  tell  someone  about  your  future 

violence  unless  you  want  them  to  stop 

It? 

5.  memory  - what  kind  of  person  Is  John? 

should  wc  take  his  anger  seriously? 

6.  culture  - what  happens  in  situations  of  this  sort 

? 

memory  structure  Inferences  used 
These  questions  started  us  looking  seriously  at  what  else 
was  going  on  In  understanding  besides  parsing*  Clearly  we  needed 
a memory  full  of  facts  about  the  world  to  do  any  sensible 
understanding*  At  this  point  our  focus  began  to  change*  The 
issues  of  representation  and  parsing  still  existed  of  course,  but 
memory  and  Inference  were  obviously  at  least  as  crucial* 

Around  this  time,  1 met  Abelson,  who  was  working  on  beliefs 
as  was  Colby*  I began  to  see  that  beliefs  had  e great  deal  to  do 
with  the  processing  of  language*  Jly  groop*ll*  began  to  attack 
this  problem  In  a number  of  ways*  Hemphll]  (1975)  began  to  work 
on  Identifying  how  parsing  was  Influenced  by  beliefs  implicitly 
referred  to  in  a text*  I,  as  usual,  concentrated  my  efforts  on 
representation*  In  particular,  it  was  necessary.  In  order  to 


*11*  By  this  time,  a number  of  students  had  begun  working  with 
me*  Goldman,  Rieger,  Flesbeck,  Hemphill  and  Weber  finished 
degrees  with  me  at  Stanford  and  were  active  In  weekly  meetings 
held  during  this  time  to  discuss  these  Issues.  A number  of  other 
students  who  didn't  finish  contributed  as  well* 


handle  the  above  example,  to  postulate  a set  of  beliefs  that 
could  account  for  our  expectations  about  an  actor's  behavior.  To 
understand  that  John  was  not  likely  to  want  to  now  sit  down  and 
be  friendly  In  the  above  example  we  needed  to  know  that  when 
you're  angry  you  don't  like  to  be  with  the  people  you  are  angry 
with.  This  was  represented  as: 

one  <==>do 

1 

I I I 

one  <=“>angry 

2 

III 

one  <=“>want 
2 

I 

one  <=*>lnteract 
1 

one 

2 

Beliefs  of  this  sort  were  useful  for  predicting  the  future 
actions  of  an  actor.  Adding  beliefs  to  the  representation 
changed  the  Idea  of  Inference  as  just  added  Information  that 
would  help  In  the  parsing  of  a sentence.  Tt  suggested  that  we 
had  to  concentrate  on  problems  having  to  do  with  the 
representation  of  information  in  memory,  and  with  the  overall 
Integration  of  incoming  data  with  a given  memory  model.  It  thus 
became  clear  that  natural  language  processing  was  a bit  of  a 
misnomer  for  our  enterprise.  What  wo  were  doing  was  not 
essentially  different  from  wliat  Colby  or  Abelson  were  doing. 
Tt»at  Is,  wc  had  to  deal  with  the  problem  of  belief  systems  in 
general.  But,  added  to  that,  was  the  problem  of  representation 


of  meaning,  knowledge,  and  memory  In  general. 


1972 


The  Integration  of  all  these  problems  caused  us  to  deal  with 
sentences  whose  meaning  was  a product  of  the  combination  of  all 
these  issues.  For  example,  "He  acts  like  Harry"  means  different 
things  if  Harry  Is  a cat,  a child  or  an  aged  man.  What  Is  the 
correct  representation  for  the  meaning  of  such  a sentence? 
Clearly  It  cannot  be  determined  In  any  way  apart  from  the  memory 
structures  Its  meaning  relies  on.  Similarly,  the  sentence  "He  Is 
doglike  in  his  devotion"  means  nothing  If  there  1 s no  belief 
about  the  devotion  or  lack  of  It  of  dogs  available  In  memory. 

In  attempting  to  catalog  the  kinds  of  beliefs  that  were 
being  used  In  natural  language  expressions,  we  came  across  two 
(Schank,  1974)  that  used  inference  quite  frequently.  These  were: 

Pay  back  In  kind. 

Pay  back  not  In  kind. 

Such  beliefs  were  used  In  sentences  such  as: 

John  threw  a hammer  at  Bill  vengefully. 

Mercifully  the  king  only  banished  the  knight 

for  killing  his  favorite  horse. 

This  kind  of  analysis  meant  that  we  could  predict  other  actions 
from  what  we  determined  was  a reasonable  Inference.  That  is  we 
could  make  Inferences  from  Inferences.  So,  'vengefully'  told  us 
that  Bill  may  have  done  something  to  John,  and  that  John  really 
wanted  to  hurt  Rill  and  so  on.  Such  Inferences  were  an  Important 
part  of  the  understanding  process. 
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The  cataloging  of  such  random  facts  was  not  within  our  view 
of  how  to  attack  a problem,  however.  Instead  we  returned  to 
attempting  to  make  more  rigorous  the  CD  representations  we  wore 
using  so  that  we  could  better  establish  what  was  within  the 
domain  of  a system  like  CD  and  what  was  outside  of  It.  To  do 
this,  we  considered  the  nature  of  the  ACTs  we  had  been  using*  At 
that  point  we  had  been  using  'trans'  and  a hodgepodge  of  others 
that  suited  us.  To  remedy  this  situation  we  looked  at  the  mental 
verbs  which  we  had,  to  this  point,  virtually  ignored .*12* 

The  significance  of  the  primitive  ACTs  for  us  was  that  we 
could  now  be  sure  that  we  had  a given  agreed-upon  representation 
for  most  of  the  sentences  we  were  dealing  with.  This  made  our 
system  usable  by  the  large  group  of  students  who  were  beginning 
to  concern  themselves  with  programming  systems  that  could 
communicate  with  each  other.  Further,  we  now  knev/  what  was  in 
the  bounds  of  the  theory  and  what  was  not.  VJe  knev'  that  to  do 
the  kind  of  work  we  were  interested  in,  a canonical  form  was 
necessary.  We  were  not  so  concerned  with  the  ultimate 
correctness  of  that  system  as  we  were  with  its  usability,  ho 
other  canonical  form  existed,  and  transformational  deep  structure 
representations,  which  were  the  major  well  known  alternative, 
neither  adequately  represented  meaning  nor  were  In  any  sense 
canonical.  The  most  Important  part  of  the  primitives  of  CD  for 


*12*  We  formed  a special  group  to  consider  this  problem, 
consisting  of  Goldman,  Rieger,  Rlesbeck  and  myself.  Eventually, 
each  of  these  students  came  up  with  one  mental  ACT  and  defended 
it  to  the  others.  This  left  us  with  three  mental  ACTs,  CONC, 
MBUILD  and  MTRANS  Invented  by  the  above  people  respectively. 
This  gave  us  a total  of  sixteen  primitive  ACTs. 
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us  then  were  that  they  facilitated  our  getting  on  to  the  more 
Interesting  problems  at  hand.  They  did  this  because  they  gave  us 
a language  In  which  to  describe  those  problems. 

1973 


The  most  Important  problem  was  Inference.  The  first  paper 
on  the  complete  set  of  primitives  we  had  (Schank, 1973)  made  that 
clear  in  its  title:  'The  Fourteen  Primitive  Acts  and  Their 

Inferences'.  The  single  most  important  fact  about  the  primitive 
ACTS  was  that  they  helped  to  organize  the  Inference  problem.  No 
primitive  ACT  meant  anything  In  the  system  at  all,  other  than  the 
conceptualizations  that  might  come  to  exist  as  Inferences  from 
It.  Primitive  ACTs  served  to  organize  the  Inference  process, 
thus  giving  us  a starting  point  from  which  to  attack  the  problem. 

We  began  to  concern  ourselves  therefore  with  two  principle 
kinds  of  Inference,  results  from  ACTs  and  enablements  for  ACTs. 
Then,  having  exhausted  the  Inferences  derivable  from  the  ACTs 
themselves,  we  began  to  attempt  to  categorize  the  kinds  of 
inferences  that  needed  to  be  made  in  general.  In  Schank  and 
Rieger  (1974)  we  delimited  the  following  kinds  of  Inference: 

1.  Linguistic  Inference  (done  before  parsing  Is  over) 

buy  - infers  "money"  as  object 
hit  - Infers  "hand"  as  object 

2.  ACT  Inference 

wtionevor  an  actor  and  object  were  present  In  a CD 
an  action  had  to  be  Inferred.  Thus  for  "I  like  books" 
"read"  Is  Inferred. 


3. 


TRANS-F.NABLE 
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For  sentences  such  as  "John  wants  a book"  It  Is 
necessary  to  Infer  an  ATRANS  which  then  enables 
an  ACT  to  take  place.  That  ACT  can  be  determined 
by  ACT  Inference. 

Besul t Inference 

For  any  given  ACT  its  results  can  be  easily 
determined.  "John  went  to  N.Y."  Implies  he  got 
there.  "Mary  gave  Bill  a book"  implies  Pill  has  It 
and  Mary  no  longer  does.  These  come  from  PTRANS  and 
ATRANS  respectively. 

Object  Affect  Inference 

Inferences  come  from  the  interaction  of  objects 

and  ACTS.  In  "John  hit  Mary  with  a rock"  we  Infer  tha 

Mary  is  damaged  and  the  rock  Is  not.  In  "John  ate  an  e 

we  Infer  that  the  egg  has  been  transformed.  These 
Inferences  come  from  the  ACT. 

Belief  Pattern  Inference 

When  we  see  "John  hit  Mary"  we  Infer  that  Mary  must 
have  angered  him  by  doing  something.  This  Inference 
Is  gotten  from  a belief.  The  belief  Is  accessed 
by  matching  a pattern  containing  "Intentional  damage" 
which  Is  written  in  CD.  This  inference  does  not  come 
from  ACTS  therefore  but  from  states. 

Instrumental  Inference 

We  can  Infer  instruments  for  ACTs  that  we  have  found. 
Thus  INGEST  implies  PTRANS  as  instrument. 

PTRANS  implies  MOVE  or  PROPEL  as  instrument  and  so  on. 

Property  Inference 

Did  Nixon  run  for  President  in  1663?  This  can  best  be 
determined  by  examining  properties  such  as  whether 
Nixon  is  alive  or  If  It  Is  election  year  before 
doing  any  exhaustive  memory  search.  The  inference 
of  those  underlying  propositions  means  that 
preconditions  for  actions  must  be  known.  That  is 
simple  when  the  actions  are  simple  ACTs.  For  concepts 
such  as  an  election  It  Is  harder. 

Sequential  Inference 

Results  of  the  combination  of  two  sentences  can  bring 
up  new  information.  Thus  "All  redheads  are  Obnoxious" 
followed  by  "Mary  has  red  hair",  makes  the  latter 
statement  more  contentive.  Such  additional  content 
depends  on  knowledge  about  Mary's  personality.  The 
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correct  Inference  requires  a memory  search. 

10.  Cnusnllty 

"John  hit  Mary  and  she  died"  or  "John  hit  Mary" 
followed  by  "John  died"  both  Imply  causality.  Such 
causality  Information  can  only  be  determined  by 
examining  the  resultatlve  properties  of  the  ACT. 

11.  Backward  Inference 

We  can  often  determine  a new  fact  by  pondering  how 
an  old  fact  came  to  be.  Thus  "John  knows  where  Mary 

Is"  Implies  that  he  saw  her  or  someone  told  him.  This 
comes  from  the  enablements  for  an  action. 

12.  Intention 

It  Is  Important  to  know  why  people  do  what  they  do. 
Thus  Intentions  and  motivations  had  to  be 
Inferred.  We  really  did  not  know  how  to  do  that. 


Rieger,  Rlesbeck,  Goldman  and  I began  to  design  a computer 
Implementation  of  these  Ideas  In  1972  which  resulted  In  the 
MARGIE  system  (Schank  et  al ., 1975) .*13*  During  the  Implementation 
of  these  Ideas  our  views  on  parsing,  generation  and  Inference 
were  altered  by  the  task  of  attempting  to  specify  precise 
algorithms  for  these  processes.  In  particular  for  our  discussion 
of  Inference,  Rieger  created  a new  classification  of  Inferences 
based  on  his  experiences  with  MARGIE  (Rieger,  1974).  These  were; 


1.  Specification 

2.  Causative 

3.  Resultatlve 

4.  Motivational 


- unmentloned  particulars 
are  Inferred 

- reasons  for  an  action 

- results  of  an  action 

- motivations  for  an  action 


*13*  Actually  this  is  not  quite  accurate.  The  MARGIE  system  was 
never  Intended  to  work  as  a whole  system,  and  It  was  not  designed 
that  way.  Jerry  Feldman  suggested  that  we  put  It  all  together. 
Prior  to  his  suggestion  we  just  had  three  unrelated  student 
projects  meant  to  cover  three  areas  of  Interest. 


Page  25 


5. 

Enablement 

- 

enablements  for  an  action 

6. 

Function 

- 

when  an  object  Is  mentioned 

Its  potential  use  Is  determined 

7. 

Enable  Prediction 

- 

what  ACT  will  be  enabled  by  ACT  or  STATE 

P. 

Missing  Enablement 

- 

why  someone  can't  do  an  ACT 

9. 

Intervention 

- 

reasons  for  actions  that  prevent  harm 

10. 

Action  Prediction 

- 

predict  ACT  from  object 

11. 

Knowledge  Propagation 

- determining  who  knows  what 

12. 

Normative  inferences 

- determining  normal  states  of  the  world 

13. 

State  Duration 

- 

bow  long  an  ACT  or  STATE  goes  on 

14. 

Features 

- 

who  can  be  expected  to  do  what 

15. 

Situation 

- 

events  are  Imbedded  In  larger  events 

16. 

Utterance  Intent 

- 

why  people  do  what  they  do  and  say  what 
they  say 

In  Schank  (1975)  we  attempted  to  further  codify  the  kinds  of 

Inferences  that  were  available  for  a given  ACT.  For  example,  we 

listed  these  rules  for  the  ACT  PROPEL: 

1 <Z 

X<~>PROPEL<— Y<—  I 

I <W 

1.  TRANS  is  implied  If  (Obj 

Is  not  fixed) 

2.  Object  In  directive  case  (Z)  Is  negatively  affected 

If  PHYSCONT  Is  present  and  sizes  are  right. 

3.  If  Z is  human  then  X may  have  been  angry  at  Z 

4.  If  Y Is  rigid  and  brittle  and  nonflxed,  and  speed 

of  Instrumental  ACT  is  great,  then  Y will  be  NEGPHYSST 
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In  general  we  noted  that  there  were  two  major  kinds  of 
Inferences : 

Forward  > what  consequences  from  an  ACT? 

Backward  > why  an  ACT  and  what  enabled  an  ACT* 

At  this  point  we  began  to  take  seriously  the  problem  of 
codifying  the  kinds  of  causal  relations  that  there  were. *14*  This 
work  was  crucial  to  the  Inference  problem  since,  we  had  come  to 
believe  that  the  major  Inferences  were  (forward)  consequences  and 
(backward)  reasons.  Thus  the  primary  task  of  the  Inference 
process  was  the  filling  In  of  causal  chains.  We  identified  four 
kinds  of  causal  links,  RESULT,  REASON,  INITIATE  and  ENABLE. 
RESULT  and  ENABLE  were  the  forward  and  backward  causal  rules  for 
physical  ACTs,  and  REASON  and  INITIATE  were  the  forward  and 
backward  links  for  mental  ACTs.  We  also  added  the  rule  that  ACTs 
could  only  result  In  states  and  only  states  could  enable  ACTs. 
This  had  the  consequence  of  making  our  causal  chains  and  thus  our 
CD  representations  both  very  precise  and  very  cumbersome.  The 
precision  was  of  course  Important  for  any  canonical  form,  but  the 
cumbersomeness  was  obviously  a problem  that  needed  to  be  dealt 
with. 


As  an  example  of  the  kinds  of  representations  we  were 
creating  by  doing  this,  consider  the  representation  that  we  nov» 
liad  for  a sentence  such  ns  "John's  cold  improved  because  I gave 


*14*  I had  wanted  to  do  this  for  some  time,  but  really  had  not 
had  the  opportunity.  In  1973  I took  a year  off  from  Stanford  and 
went  to  Lugano,  Switzerland  where  a new  Institute  was  starting 
up,  taking  Rlcsbeck  and  Goldman  with  me.  We  had  little  to  do 
there  but  think  and  I was  able  to  finish  the  book  on  MARGIE  and 
start  working  on  issues  that  I had  not  had  time  for  before. 


r 


•■»»re.v 
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him  an  apple." 


o D I— >John 

I <«>PTRANS< apple  < 1 


I >LOC  (John) 

apple 


I I I E o D I >John 

John  <““»>INGEST  < apple  < j 

POT  I < 

III  I o I >I  want  to  eat  the  apple 

John  <“«“>MBUILD  < | 

I < 

I I I R D I >John 

John  <“««>INCEST  < apple< j 


John  <— >1 >HEALTH  (XTY) 

P 1 <HEALTH  (X) 


One  of  the  advantages  of  all  this  detail  aside  from  those 
already  mentioned  Is  that  It  provided  a facility  for  tying 
together  sentences  In  a text.  Thus,  a paragraph  will  frequently 
consist  of  a scries  of  conceptualizations  that  can  be  related  by 
their  Implicit  causal  connections. 


1974 


We  began,  therefore,  to  work  on  the  problem  of  representing 
text.  This  was,  after  all,  the  major  Issue  all  along.  We  were 
not  particularly  Interested  In  Isolated  sentences  out  of  context. 
Such  sentences  were  probably  the  root  of  many  of  the  solutions 
and  the  problems  with  those  solutions  found  by 
transformationalists  and  computational  linguists.  People  do  not 
understand  sentences  In  a null  context.  Why  then  did  our 
theories  try  and  deal  with  out  of  context  sentences?  The  answer 
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was  obviously  that  tills  was  thought  to  a simplification  that 
would  facilitate  research.  But  the  problem  was  really 
significantly  changed  by  this  supposed  simplification.  Certainly 
parsing  sentences  In  context  is  a more  reasonable  problem  with 
respect  to  word  sense  disambiguation  than  is  parsing  out  of 
context . 

We  had  never  dealt  with  texts  of  more  than  one  sentence 
before  because  we  just  did  not  know  how  to  represent  them.  Now, 
with  the  Idea  of  causal  chains,  we  could  tie  together  texts  In 
terms  of  their  causal  relations.  Such  a tying  together,  when 
attempted  on  real  texts  (Schank,  1975)  helped  to  explain  certain 
memory  results  (particularly  those  of  Bartlett,  1932).  Now  we 
had  a theory  that  said  that  a crucial  piece  of  Information  had 
many  causal  connections  and  an  irrelevant  piece  of  information  no 
causal  consequences. 

The  work  on  causal  connectedness  gave  us  a theory  that  was 
helpful  In  explaining  problems  of  forgetting  and  remembering,  and 
helped  tie  together  text.  However,  It  could  not  explain  how  to 
tie  together  texts  whose  parts  were  not  relatable  by  chains  of 
results  and  enablements.  Something  else  was  needed  for  those 
sltua  tlons . 

The  something  else  was  obvious  once  we  thought  about  It. 
The  answer  was  scripts. *15*  That  Is,  scripts  arc  really  Just 
prepackaged  sequences  of  causal  chains.  Some  causal  chains  are 
used  so  often  that  we  do  not  spell  out  enough  of  their  details 
for  an  understander  to  make  the  connections  directly.  Scripts 
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ore  o kind  of  key  to  connecting  events  together  thnt  do  not 
connect  by  their  superficial  features  but  rather  by  the 
rememberance  of  their  having  been  connected  before.  The 
prototypical  script  we  chose  was  to  describe  what  goes  on  In  a 
restaurant.  In  a restaurant  we  cannot  Infer  from  entering  a 
restaurant  the  causal  connection  to  either  ordering  or  paying. 
Because  speakers  assume  you  know  that  they  do  not  bother  to 
mention  It.  There  Is  a causal  chain  there,  but  inferring  It  bit 
by  bit  Is  Impossible,  so  scripts  are  necessary. 

1975 


We  set  about  testing  our  assumptions  about  how  scripts  %K>uld 
facilitate  the  processing  of  connected  text  by  building  SAM 
(Script  Appller  Mechanism) .*16*  SAM  became  a kind  of  Inference 
maker  because  «d)at  It  was  doing  was  filling  out  the  specific 
Implicit  events  In  a causal  chain  describing  a static  situation. 


Culllngford  (1977)  described  the  Inferences  that  SAM  made  as: 


*15*  The  word  script  was  originally  used  by  Abelson  for  something 
different  than,  but  related  to,  the  current  notion  of  a script. 
The  concept  of  a script  was  Invented  very  shortly  after  I arrived 
at  Yale  from  Lugano.  Abelson  and  I were  discussing  Issues  of 
mutual  Interest  as  soon  as  I arrived.  At  one  time  or  another, 
Rieger,  David  Levy,  and  Allan  Collins  were  also  present. 
Minsky's  notion  of  a frame  was  at  that  time  known  by  us,  and  It 
had  some  Influence  In  the  finalization  of  the  notion. 

*16*  As  soon  as  I arrived  at  Yale,  I assembled  a group  of 
students  Interested  In  AI.  In  January  1975  they  began  to  %iork 
with  me.  We  thought  for  a while  In  seminars  about  what  scripts 
were  like,  and  In  early  May  began  to  put  together  SAM.  The 
initial  version  worked  about  six  weeks  later  just  In  time  to  be 
shown  at  TTNLAP  I.  The  group  thnt  put  SAM  together  was  Richard 
Culllngford,  Richard  Proudfoot,  Walter  Stutzman,  Wendy  Lehnert, 
Carry  Do  Jong,  and  Chris  Rlesbeck. 


1.  Reference  specification 

2.  Causal  Chain  connection 
as  part  of  Script  Appller 

Other  kinds  of  Inferences  were: 

1 - Immediate  results:  in  order  to  match  PTRANS 

to  the  PATTERN  - IS  LOC 

inferences  about  result  were  needed  "Welcomed  at 
Peking  Airport"  generates  PTRANS  which  can  then 
match  $VIPVISIT 

2 - Mental  ACT 

if  X says  Y then  Y — thus,  pronounced  dead 

Implies  that  a person  Is  dead. 

service  good  Implies  customer  knows  it. 

Thus  activates  $TIP 


3 - Location 

a location  described  by  one  method  Is  the 
same  as  that  described  by  another 

"customer  sat  down" 

"waiter  went  to  the  table" 

need  to  infer  that  the  seat  Is  near  table 

Enclosure:  John  went  to  the  hospital. 

He  was  treated  In  the  Emergency  Room. 

need  to  Infer  that  Emergency  Room  Is  In  hospital 

4 - Movement 

"John  picked  up  a magazine.  Then  he  went  Into 
the  living  room." 

To  answer  "Where  Is  the  magazine?"  requires 
knowing  that  small  objects  move  with  you. 
(Imagine  "table"  substituted  for  nempaper) 

We  began  to  wonder  about  where  scripts  came  from.  In 

thinking  about  this  we  came  up  with  the  Idea  that  plane  gave  rise 

to  acrlpts  and  that  goals  gave  rise  to  plana*  Coincidentally 

Meehan  was  developing  a story  generator  that  served  as  a vehicle 

for  developing  our  ideas  about  plans  and  goals. *17* 
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Meehan's  TALESPIN  made  Inferences  In  the  course  of  telling  a 
story.  In  order  to  keep  track  of  the  world  model*  Meehan  (1976) 
found  that  Rieger's  16  classes  were  of  little  use  In  this  task 
because  the  most  Interesting  question  was  what  is  affected  when  a 
fact  enters  memory?  Meehan  had  to  keep  updating  his  world  model 
every  time  he  generated  a new  fact.  This  meant  making  Inferences 
about  the  consequences  of  every  fact  that  he  generated  and  using 
those  consequences  to  affect  the  continuation  of  the  story. 
Meehan  found  the  following  kinds  of  inferences  useful  for  his 
task: 


1.  CONSEQUENCES 

ATRANS > POSS 

FIXED > (let  go  — >fall) 

INGEST  > object  gone 

person  satisfied 

2.  REACTIONS 

What  people  will  feel  about  things  and  people 

ATRANS  what  you  want >dlsllke 

THREATEN  > person  will  do  ACTION  If  afraid 

HUNGER  > actor  %rants  to  fix  It 

LOC 

(underwater)  > get  out 

flattery  > like  and  trust 

1976-1978 


The  last  three  years  have  found  us  developing  the  system  of 
plans,  goals,  themes  and  scripts  for  use  In  understanding 
systems.  This  work  produced  many  working  systems  (Carbonell , 1978 
WllensVy,  1978,  DeJong,  1978)  and  has  greatly  broadened  our  ideas 


*17*  Meehan  had  been  working  on  a story  generator  of  a different 
sort  when  I arrived  at  Yale.  The  plans  and  goals  *wrk  that  we 
were  doing  forced  him  to  reconsider  his  problem  and  approach  It 
from  a different  angle. 


I 

I 

L 
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about  inference.  We  now  believe  the  following; 

There  are  a great  many  possible  levels  of  description.  Fach  of 
these  levels  Is  characterized  by  Its  own  system  of  primitives  and 
conceptual  relationships.  Inferences  occur  at  each  of  these 
levels.  Thus,  for  every  set  of  primitives  there  are  a set  of 
Inferences  that  apply  to  them.  These  levels  have  been  described 
in  Schank  and  Abelson  (1977)  and  will  not  be  dealt  with  In  any 
detail  here.  We  currently  use  the  following  levels  and 
Inferences  on  those  levels; 

1 - Micro  CD 

All  events  in  a story  can  be  connected  at  a level  where 
every  event  Is  connected  to  the  events  chat  follow  from  It,  and 
to  the  states  enable  It  subsequent  events . This  produces  a very 
detailed  causal  chains  made  up  of  the  events  and  states  that  vrere 
actually  mentioned  in  the  text  as  well  as  those  that  had  to  be 
inferred  in  order  to  complete  the  chain.  Thus,  the  Causal  Chain 
made  by  the  low  level  expression  of  facts  Is  one  part  of 
understanding.  Thus,  in  order  to  read  a magazine,  you  must; 
ATRANS  it;  OPEN  It;  ATTEND  to  it;  and  MTRANS  from  it.  When 
any  one  of  these  events  Is  discerned  the  others  must  be  inferred. 

2-Macro-CD 

Another  type  of  causal  chain  exists  at  the  Macro-CD  level. 
There,  events  connect  to  other  states  and  events  in  the  same  way 
as  they  did  at  the  micro-CD  level  but  the  level  of  description  Is 
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different.  Thus,  going  to  Boston  enables  eating  In  a Boston 
restaurant  at  the  macro-CD  level.  But,  at  the  mlcro-CD  level, 
the  locations  would  have  to  be  further  specified.  Actually  going 
to  Boston  results  in  being  in  Boston  which  enables  beginning  to 
look  for  and  go  to  a restaurant.  This  latter  level  of 
description  can  regress  in  infinite  detail  where,  for  example, 
walking  Is  enabled  by  putting  one  foot  in  front  of  the  other. 
Thus,  the  level  of  detail  of  Inferences  Is  extremely  important 
and  is  dependent  on  the  purposes  the  understander  has  In  mind. 

Thus,  there  are  two  levels  of  causal  chains  that  apply  In  the 
magazine  situation. 

ATRANS 

III  r 

POSS 

III  E 

ATTEND 

111  rE 
MTRANS 

This  is  MACRO-CD. 


MICRO-CD  is  concerned  with  opening  the  magazine,  holding  It 
turning  the  pages  etc.  Each  of  those  ACTs  also  uses  causal 
chains  but  at  a much  more  detailed  level.  Neither  one  of  these 
levels  of  description  Is  more  correct  than  the  other. 


Pagp  34 


For  causal  chaining  then,  the  needed  Inference  types  are: 

What  Knables 
What  Results 
What  arc  Reasons 
What  Initiates 

These  apply  at  both  at  the  macro  level  and  at  the  micro  level. 


3.  Filling  in  Missing  Information 

For  every  object  and  person  we  he.ir  about  we  are  always  tracking 
where  they  are;  the  state  they  are  in;  what  they  know  and 
believe;  and  how  they  feel.  ^11  these  Inferences  are  possibly 
appropriate  at  any  given  time.  Thus,  other  kinds  of  Inference 
types  that  are  necessary  are: 

Locational  specification 

■ .* 

Object  speclflcaltlons 
F,motional  specifications 
Belief  Specifications 


4.  Scripts 

Scripts  are  an  important  part  of  the  understanding  process. 
Thus,  the  inferring  of  the  presence  of  scripts  and  of  the 
unstated  parts  of  scripts  is  an  Important  part  of  the 
understanding  process.  The  following  kinds  of  inference  are 
significant : 

filling  in  missing  causal  chains  in  a script 
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Inferring  wl’ar  script  is  being  userf 

Inferring  what  unstated  script  was  used  instrumental ly 

Thus,  when  we  hear  that  'John  robbed  the  liquor  store,'  It  Is 
appropriate  to  ask  how  he  got  there,  how  he  got  In,  where  he  got 
his  weapon,  and  so  on.  Such  inquiries  are  a part  of  the 
inference  process  since  it  Is  only  by  knowing  what  we  don't  know 
that  we  can  seek  to  infer  It* 

One  of  the  main  problems  with  reference  to  inferences  about 
scripts  is  the  question  of  why  Is  a script  being  pursued.  This 
leads  to  the  problem  of  Inferring  plans. 

5.  Plans 


For  any  given  event,  it  is  often  Important  to  know  the 
motivations  and  Intentions  of  the  the  actors  in  that  event.  This 
means  knowing  the  plans  being  pursued  by  an  actor.  Thus  It  is 
necessary  to  make  the  following  kinds  of  inferences: 

Inferring  the  planbox  being  used 
Why  was  a particular  planbox  chosen? 

Inferring  facts  about  an  actor  given 
his  choice  of  plans  S planboxes 
Inferring  other  plans  an  actor  is  likely 
to  pursue  to  get  his  goal 

Predictive  Inferences  about  future  planbox  choices 
What  goal  is  he  operating  under? 
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This  last  inference  leads  to  another  class  of  Information 
that  spawns  new  inferences* 

6.  Coals 

Detecting  the  presence  of  a goal  causes  the  following  goal 
based  Inferences  to  be  made: 

Wliy  was  this  goal  chosen? 

What  is  In  conflict  with  It? 

Can  It  be  subsumed? 

Given  this  goal,  what  other  goals  can  we  infer? 

Under  what  circumstances  will  it  be  abandoned? 

Actually  these  Inference  types  represent  only  the  tip  of  the 
numerous  kinds  of  goal  based  Inferences  that  have  been  isolated 
by  Wllensky  (1978)  and  Carbonell  (1978). 

Here  again  since  goals  are  dominated  by  themes,  detecting 
wliat  theme  is  present  and  making  the  approprite  Inferences  Is 
necessary . 

?.  Themes 

The  theme  based  Inferences  Include  finding  out: 

What  goals  will  be  generated  next? 

What  themes  are  likely  to  coexist  with  the  given  one? 

Are  there  any  conflicts  In  themes? 

How  might  theme  conflicts  that  are  detected  be  resolved? 


Where  did  this  theme  come  from? 
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The  Inference  types  we  have  used  are  rather  similar  whether  we 
are  referring  to  scripts,  goals,  plans,  themes,  or  whatever. 
Inside  Conceptual  Dependency  structures,  or  knowledge  structures 
or  our  recently  Invented  triangular  structures  (Schank  and 
Carbonell , 1978),  or  probably  any  reasonable  representation  of 
knowledge,  the  following  general  Inference  rules  apply: 


i 


! 


1.  SPECIFICATION;  Given  a piece  of  an  event,  what  else  can  be 
speclfleJ  about  the  rest  of  the  pieces? 

2.  MOTIVATION;  Why  did  an  event  happen?  Why  this  event  and  not 
another?  What  did  the  actor  believe  he  vras  doing? 

3.  ENABLEMENT:  What  was  necessary  for  the  event  to  occur? 

4.  RESULTS:  What  are  the  results  or  effects  of  this  event? 

5.  STRUCTURE  : What  higher  level  structure  does  this  fit  In? 

6.  OTHER  EVENTS:  What  other  events  are  known  to  cooccur  with 
this  event?  What  could  not  have  happened  if  this  event  happened? 


These  six  Inference  types  then  are  what  we  have.  Scripts, 
plans  and  so  on  fit  In  as  events  In  the  above  description.  Thus, 


we  can  ask  for  SPECIFICATION,  MOTIVATION,  ENABLEMENT,  RESULTS 
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STRUCTURE  and  OTHER  EVENTS  for  a script,  a plan,  a goal,  or  a 
theme  or  probably  any  other  higher  level  structure  we  are  likely 
to  Invent* 

Inference  then.  Is  the  fitting  In  of  new  Information  Into  a 
context  that  explains  It  and  predicts  other  facts  that  follow 
from  It*  Since  these  explanations  can  occur  at  many  levels 
Inference  Is  a very  complex  problem  and  one  we  expect  to  continue 
working  on  In  an  attempt  to  find  out  how  people  understand  and 
how  computers  could  understand* 

Overview 

I have  attempted  here  to  demonstrate  how  our  Ideas  evolved 
and  why  they  are  where  they  are  today*  Since  this  theory 
evolution  Is  ongoing  It  should  be  clear  that  the  conclusions  we 
have  reached  about  Inferences  here  are  probably  also  just 
stopping  points  In  the  evolution  of  a theory*  Nevertheless  there 
arc  some  things  we  can  conclude  from  all  this*  In  particular, 
there  are  patterns  from  which  we  can  get  a glimpse  of  the  future* 

As  I have  stated,  this  work  started  out  as  a linguistic 
theory,  albeit  one  with  a computer-based  bias*  Linguists  have 
explicitly  rejected  It  as  a possible  linguistic  theory  (see  for 
example  Dresher  and  Hornsteln,  1976)*  In  one  sense  they  are 
right*  The  phenomena  we  have  become  Interested  In  over  the  years 
are  not  particularly  phenomena  of  language  per  so*  Rather,  they 
are  phenomena  having  to  do  with  the  processing  of  language  in 
general  and  the  Issue  of  the  representation  of  knowledge  In 
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particular.  Thus,  as  we  have  moved  away  from  linguistics  over 
the  years,  wc  have  become  more  involved  with  psychology. 

At  the  same  time  as  this  work  was  going  on,  the  field  of 
Artificial  Intelligence  has  been  evolving  too.  When  I first 
arrived  at  the  Stanford  AI  lab,  the  major  issues  in  AT  were 
theorem  proving,  game  playing  and  vision.  Natural  language  was 
not  considered  to  be  a serious  part  of  AI  until  Wlnograd  (1971) 
presented  the  AI  community  with  SHRDLU.  This  work  contributed 
substantially  to  the  evolution  of  AI.  The  major  concern  of  AI 
would  now  seem  to  be  the  issue  of  the  representation  of 
knowledge,  which  of  course  makes  the  work  In  natural  language 
processing  quite  central. 

In  the  future  I expect  many  of  the  relevant  fields  will 
begin  to  become  less  separate.  AI  must  come  to  terms  with  the 
fact  that  It  Is  concerned  with  many  Issues  that  are  also  of 
Interest  to  philosophers.  I hope  that  the  cooperation  will  be  of 
more  use  than  was  the  head  butting  that  has  gone  on  between  AI 
people  and  linguists.  (Although  this  too  has  changed  as  the  more 
liberal  forces  in  linguistics  have  become  both  stronger  and  more 
Interested  In  AI.)  Also,  the  Interaction  between  psychologists 
and  AI  people  should  continue  to  flourish.  The  work  of  Power  ct 
nl  (197P)  and  Smith  (I97f>)  has  already  served  to  bolster  the 
relationship  between  our  group  and  cognitive  psychology. 

And  what  will  happen  to  our  theories  In  the  future?  I can 
only  say  that  many  of  our  Ideas  on  parsing,  the  separation  of 
inference  from  other  processes,  generation,  and  memory  are 
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rapidly  changing.  We  will,  of  course,  continue  to  use  the  same 
methodology  of  the  free  form  speculation  approach  to  theory 
building,  modified  by  our  experiences  testing  out  these  theories 
on  the  computer. 


’ 
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